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Abstract 

We study Pippenger's model of Boolean networks with unreliable gates. In this 
model, the conditional probability that a particular gate fails, given the failure 
status of any subset of gates preceding it in the network, is bounded from above 
by some e. We show that if a Boolean network with n gates is selected at random 
according to the Barak-Erdos model of a random acyclic digraph, such that the 
expected edge density is crt~^ log n, and if e is equal to a certain function of 
the size of the largest reflexive, transitive closure of a vertex (with respect to a 
particular realization of the random digraph), then Pippenger's model exhibits a 
phase transition at c = 1. Namely, with probability 1 — o(l)asn— i>oo, we have the 
following: for < c < 1, the minimum of the probability that no gate has failed, 
taken over all probability distributions of gate failures consistent with Pippenger's 
model, is equal to o(l), whereas for c > 1 it is equal to exp {^—jf-^zri^ + 
We also indicate how a more refined analysis of Pippenger's model, e.g., for the 
purpose of estimating probabilities of monotone events, can be carried out using 
the machinery of stochastic domination. 

Keywords and Phrases: Boolean network, Lovasz local lemma, phase tran- 
sition, probabilistic method, random graph, reliable computation with unreliable 
components, stochastic domination. 
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1 Introduction 



The study of phase transitions in combinatorial structures, such as random graphs 013113 is a sub- 
ject at the intersection of statistical physics, theoretical computer science, and discrete mathematics 
[H El 13 El HSl Oni OH lini ■ The key idea behind this study is that large combinatorial structures 
can be thought of as systems consisting of many locally interacting components, in direct analogy 
to the kinds of systems within the purview of statistical mechanics. A phase transition, then, is a 
phenomenon that takes place in certain kinds of such systems in the limit of an infinite number of 
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components, and corresponds qualitatively to a change in some global (macroscopic) parameter of 
the system as the local parameters of the components are varied. 

Boolean networks with gates subject to probabilistic failures fall naturally into the category of 
systems just described. The possibility of a phase transition arises here, for instance, when one 
associates a probability of failure with each gate of such a network, and then looks at the maximum 
of the probability that the network deviates (outputs an incorrect result), taken over all possible 
assignments of inputs to the network, in the limit of an infinite number of gates. The theory of 
Boolean networks with unreliable gates can be traced back to the seminal work of von Neumann ^22| , 
who considered the simplest case, namely when each gate in the network fails with fixed probability 
£ independently of all other gates — we will refer to this set-up as the e-independent failure model. 
Von Neumann's initial work was developed further by Dobrushin and Ortyukov jlHll2j. Pippenger 
[nSlj Feder [T^, Pippenger, Stamoulis, and Tsitsiklis and Gacs and Gal ^^1) to name just a 
few. (It should be mentioned that in Ref. |3S] the authors pointed out several technical flaws of 
[TT] and presented their own proof of a weaker result; Gacs and Gal JHI later developed methods 
to recover the full result claimed in ^J.) Now we will summarize relevant notions and ideas in a 
more or less narrative fashion; the requisite details will be supplied in Section [21 

One of the key results obtained by von Neumann [32] was the following: if the probability e 
of gate failure is sufficiently small, then any Boolean function can be computed by a network of 
unreliable gates such that the probability of error is bounded by a constant independent of the 
function being computed. On the other hand, since in this model a Boolean network is no more 
reliable than its last gate, the probability of error can get arbitrarily close to one if the probability 
of gate failure is sufficiently large. (This is, in fact, one possibility for a phase transition in Boolean 
networks with unreliable gates, which we have alluded to in the preceding paragraph.) 

However, as pointed out by Pippenger |31j, the model of independent stochastic failures has the 
following significant drawback. Suppose that, within this model, a network is shown to compute a 
Boolean function / with probability of error at most 5, when the gate failure probability is equal to 
e. Then it cannot be guaranteed in general that the same network will compute / with probability 
of error at most 5 when the gate failure probability is smaller than e' . In particular, such a network 
may not even compute / at all in the absence of failures! This is due to the fact that gates which 
fail with a fixed and known probability can be assembled into random-number generators that 
would output independent and nearly unbiased random bits. These random- number generators 
can, in turn, be used to implement a randomized algorithm that would correctly compute / with 
high probability. However, if one were to replace the outputs of the random-number generators 
with some fixed constants, then that algorithm would be very likely to produce meaningless results. 
Another observation made by Pippenger was the following. In complexity theory of Boolean circuits 
j44j . a theorem due to Muller says that, given any two finite, complete bases B and B' of Boolean 
functions, a network over B that computes a function / can be realized as a network over B' with 
size and depth differing from those of the original circuit by multiplicative constants that depend 
only on B and B'. It is not immediately clear under what conditions such an invar iance theorem 
would hold for networks with unreliable gates. 

In order to overcome these objections, Pippenger proposed in [HI] a more general model of 
Boolean networks with unreliable gates. Gate failures under this model are no longer independent, 
but instead are such that the conditional probability of any gate failing, given the status (failed 
or not) of any set of gates preceding it, is at most e. In Pippenger's terminology this model 
is called e-admissible. It is immediately evident that the e-admissible model subsumes the e- 
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independent one. It also follows from definitions that a network that computes a function / reliably 
for all probability distributions of gate failures within the e-admissible model, will continue to do 
so under the e'-admissible model for any < e' < e. Another key achievement of Pippenger's paper 
|34j is the proof of a Muller-type invariance theorem for Boolean networks with unreliable gates, 
in which the e-admissible model plays an essential role. 

The contribution of the present paper consists mainly in showing that Pippenger's e-admissible 
model, applied to Boolean networks drawn at random according to a certain model of random 
directed acyclic graphs, exhibits a phase transition in terms of the minimum probability of the 
failure-free configuration as the network's wiring pattern evolves from "sparse" to "dense." The 
paper is organized as follows. In Section[2lwe fix definitions and notation used throughout the paper 
and collect some preliminaries on graphs. Boolean networks, and the formalism used in Pippenger's 
paper Our main result — one concerning the phase transition — is proved in Section |21 Then, 
in Section m we use the machinery of stochastic domination to carry out a systematic analysis of the 
more delicate features of Pippenger's model. We close with some remarks in Section [S] concerning 
directions for future research. Finally, in the Appendix we prove a certain theorem which, though 
somewhat tangential to the matter at hand, is closely related to some mathematical techniques and 
concepts used in this paper. 

2 Preliminaries, definitions, notation 

2.1 Graphs 

In this paper we deal exclusively with directed acyclic graphs (or acyclic digraphs). Given such a 
graph G = (V^E), we will follow standard practice of denoting by v{G) the number of vertices of 
G and by e(G) the number of edges. Any acyclic digraph has at least one vertex of in-degree zero. 
We will denote by G = {V , E) the graph obtained from G by deleting all such vertices along with 
all of their outgoing edges. 

Let us define the out-neighborhood of a vertex i G V as the set N{i) := {j G V : G E}, 

and the closed out-neighborhood as N{i) ■= N{i) U {i}. If vertex j can be reached from vertex i by 
a directed path, we will write i j (or i -^g j whenever we need to specify G explicitly). The 
transitive closure of a graph G is the graph G* = {V*, E*) with V* = V and E* = {{i,j) : i j}- 
The transitive closure of a vertex i is the set T{i) = {j G V : {i,j) G E*}; the set T*{i) = T{i)U{i} is 
called the reflexive, transitive closure (RTC, for short) of i. Note that T^*\i) is simply the (closed) 
out-neighborhood of the vertex i in G*. It is also convenient to partially order the vertices of G as 
follows: for i,j distinct, we will write i ^ j if z j, and require that i ^ i for each i. In this way, 
^ is simply the reflexive closure of the asymmetric transitive relation 

An important role in this paper will be played by the random acyclic digraph introduced by 
Barak and Erdos 0. It is obtained from the standard undirected random graph G{n,p) jS} [7| l25] 
by orienting the edges according to the natural ordering of the vertex set [n] , and will henceforth 
be denoted by Gci{n,p). 

2.2 Boolean networks 

A Boolean function is any function / : — > B, where B := {0, 1}. A set of Boolean functions 
is referred to as a basis. In particular, we say that a basis B is complete if any Boolean function 
can be realized by composing elements of B. Let ;B be a finite complete basis. A Boolean network 
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(or circuit) N over B is an acyclic digraph G with a specially designated vertex of out-degree zero 
(the output of N), such that each vertex of G is labelled by some Boolean function ip € B of its 
immediate predecessors, and each vertex in V\V is labelled either by a Boolean variable (these 
vertices are the inputs of N) or by a constant or 1. Whenever there is a need to specify the 
network N explicitly, we will write, e.g., Gn = (VN,i?i\i), etc. We will refer to the graph Gn as 
the gate interconnection graph of N. Given a network N with s input vertices, we will assume the 
latter to be ordered in some way, and therefore Xi, 1 < i < s, will denote the Boolean variable 
associated with the ith input vertex. For any assignment (xi, . . . ,Xs) G of values to the inputs 
of the network, the value of each vertex can be computed recursively in the obvious way, namely 
by evaluating the Boolean function labelling it on the values of its immediate predecessors. We 
then say that the network computes a Boolean function / : — > B if, for any (xi, . . . ,Xs) G B'^, 
the value of the output vertex, which we will denote by N(a;i, . . . ,Xs), is equal to f{xi, . . . ,Xs). 

Let us associate with a network N the measurable space (JInj-^n)) where := B^'^ and 
is the set of all subsets of Then the occurrence of failures in the gates of N is described by a 
probability measure fi on (r2N,.FN) or, equivalently, by a family {Xi : i G Vpg} of B-valued random 
variables, where Xi is the indicator function of the event 

Ai := {gate i fails} = {x G M^"" : Xi = 1}, (2.1) 

and the equivalence is, of course, given by 

P(/\(Xi = l)) =/x(f| Ai) FG^N. (2.2) 

From now on, given a probability measure /i, we will denote probabilities of various events by //(•) 
or by P^(-)i or sometimes by just IP(-)! whenever the omission of the underlying measure is not 
likely to cause ambiguity. 

Following Pippenger [211, define a probabilistic failure model (or PFM, for short) as a map M 
that assigns to every Boolean network N a compact subset M(N) of the set P(N) of all probability 
measures on (0|\i,J^m). One typically works with a family {Mg : < e < 1} of PFM's, where 
e can be thought of as a local parameter describing the behavior of individual gates; to give a 
simple example, the e-independent PFM is the map that assigns to each network N the product 
measure vr^ := IlieVN where each vl is a copy of the Bernoulli measure u with z^(l) = e. Given 
such a family M := {M^}, a network N with s inputs and a Boolean function / : B* ^ B, we say 
that N (e, 5)- computes f with respect to M if 

max sup ¥{N{xi,. . . ,Xs) ^ f{xi,. . . ,Xs)) < 6. (2.3) 
{xi,...,xs)£n= ^eMe(N) 

The maximum in Eq. (|2.3() exists owing to the finiteness of B* and to the compactness of M^{N). 
Whenever the family M contains only one PFM M, we will assume that the underlying parameter 
e is known and fixed, and say that N (e, (5)-computes / with respect to M. 

Consider a pair of PFM's, M and M' . In the terminology of Pippenger [Slj, M is more stringent 
than M' if, for any network N, M(N) D M'(N). We wiU also say that M' is less stringent than M. 
Thus, if one is able to show that a network N (e, 5)-computes a function / with respect to a PFM 
M, then the same network will also (e, (5)-compute / with respect to any PFM M' less stringent 
than M. 
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We would also like to comment on an interesting "adversarial" aspect of the PFM formalism 
(see also Ref. [SSI)- Let us fix a family {M^} of PFM's. We can then envision the following 
game played by two players, the Programmer and the Hacker, with the aid of a disinterested third 
party, the Referee. The Referee picks a constant eq S (0, 1) and announces it to the players. The 
Programmer picks a Boolean function / and designs a network N that would compute / in the 
absence of failures. He then presents N to the Hacker and lets him choose (a) the input to N and 
(b) the locations of gate failures according to M^^. We assume here that the Hacker possesses full 
knowledge of the structure of N. The Hacker's objective is to force the network to {eq, (^)-compute / 
with 6 > 1/2, and the Programmer's objective is to design N in such a way that it (eo, (^O'^omputes 
/ with 6' < 1/2, regardless of what the Hacker may do. 

2.3 Pippenger's model 

Now we state the precise definition of the e-admissible PFM of Pippenger [34j . alluded to in the 
Introduction. Given a network N, let M^i^) be the set of all probability measures fi € 'P(N) that 
satisfy the following condition: for any gate i G Vn and for any two disjoint sets Y,Y' C VN\r*(i), 
such that 

^( n n n ^) ^ 0' (2.4) 

j&Y jeY' 

we have 

^i{Ai\f] Ajn f]A])<e. (2.5) 

According to definitions set forth in Section f2. 21 we will have a PFM N i-^ A4e(N) if we prove that 
A^e(N) is a compact set. This is accomplished in the lemma below (incidentally, this issue has not 
been addressed in Pippenger's paper ^34 ). 

Lemma 2.1 The set ^^^(N) is compact in the metric topology induced by the total variation dis- 
tance 

d(/z,/x') := sup HA)-ii'{A)\. (2.6) 

Remark 2.2 The topology induced by the total variation distance is actually a norm topology, 
where the total variation norm is defined on the set A4±{N) of all signed measures on (ri^ii -^n) by 

||;u|| := sup \fi{A)\. (2.7) 

Furthermore, 'P(N), obviously being closed and bounded with respect to the total variation norm, 
is a compact subset of 

Proof. Since V{H) is compact (see Remark above), it suffices to show that A4s{N) is closed. 
Suppose that a sequence {fJ-n} in A4e(N) converges to fi in total variation distance. Let us adopt 
the following shorthand notation: for any two disjoint sets Y,Y' C Vn, let 

Cy,y' ■■= r\Ajn fl A^. (2.8) 

j&Y j€Y' 
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Fix a gate i. Let Y^Y' C VN\r*(i) be disjoint sets such that ^{Cy;y') 7^ 0. Then we can find a 
subsequence {/iric}) such that each finaiCyx') nonzero as weh. By e-admissibility, we have the 
fohowing estimate: 

H{Ai\CY,Y') < HMCy,Y') - t^nM\CY^Y')\ + t^nAMCvX') 

< \fl{A\CY,Y') - f^nM\CY,Y')\ + e. (2.9) 

We can further estimate the first term on the right-hand side of (|2.9)) : 

ti{Ai n Cy,y' ) ^n, (Ai n Cy,y' ] 



\n{Ai\CY,Y') - fJ'nc{A\CY,Y' 
1 



< 



KCy,y') 



tJ'{CY,Y') fJ'nciCY,Y') 

n Cy,y') - iJ-n^Ai n Cy,y')\ 



+ 77; ^ Mn<,(Cy,y') - h{Cy,Y')\ 

< l^\ d{fl,flnj. (2.10) 

Combining (|2.9|) and 1)2.10(1 and taking the limit along n^, we obtain iJ,{Ai\CY,Y') ^ £• Thus ^^^(N) 
is closed, hence compact. □ 

As we have mentioned earlier, Pippenger |i34j has termed the PFM N 1— > MsiN) e- admissible. We 
will also abuse language slightly by referring to individual probability measures /i € 7We(N) as 
e-admissible. 

It is easy to see that M.£{H) containts all Bernoulli product measures tt^, with < e' < e, 
as well as all product measures ni6t4i ^^^^ < < e. Furthermore, it follows directly from 
definitions that M.^i{H) C Ale(N) for < e' < e. That is, the e-admissible PFM is more stringent 
than the e'-admissible one. Therefore, when e' G [0,e], a network that (e, 5)-computes a function 
/ under the e-admissible model will also (e', 5)-compute the same function under the e'-admissible 
model and, in particular, when the gate failures are distributed according to vr^j. 



3 The phase transition 
3.1 Motivation and heuristics 

Our main result, to be stated and proved in the next section, is formulated in terms of the probability 
of the failure-free configuration in a network of unreliable gates, under the e-admissible model of 
Pippenger. As we shall demonstrate shortly, this quantity does not depend on the particular 
function being computed, but only on the size and the structure of the gate interconnection graph 
associated to the network. 

Given a Boolean network N, let us consider the quantity 

inf ^( n (3-1) 

The set A^e(N) is compact by Lemma 12.11 and the expression being minimized is a continuous 
function of jj, with respect to total variation distance. Thus, the infimum in is actually 
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attained, and a moment of thought reveals that this quantity depends only on the structure of the 
gate interconnection graph of N, but not on the specific gate labels or on the identity of the output 
vertex. Therefore, given an acyclic digraph G = {V, E), let us define F^(G) as the quantity ()3.1|) for 
all networks N whose gate interconnection graphs are isomorphic to G, modulo gate labels and the 
identity of the output vertex. In the same spirit, let us denote by A4f the set of all e-admissible 
probability measures on the measurable space {^g,^g) where, as before, '■= and !Fg is the 
set of all subsets of Qq- Then we can write 

F,(G):= ini n{f]A). (3.2) 

Our motivation to focus on Ff,[G) is twofold: firstly, we are able to gloss over such details as the 
function being computed or the basis of Boolean functions used to construct a given network, and 
secondly, -Fe(G) can also be used to obtain lower bounds on probabilities of other events one would 
associate with "proper" operation of the network (such as, e.g., the probability that the majority 
of gates have not failed [HI]). 

In order to get a quick idea of the dependence of -Fe(G) on the structure of G, we can appeal to 
the Lovasz local lemma |14j or, rather, to a variant thereof due to Erdos and Spencer JH]- (See also 
Alon and Spencer ^ and Bollobas [7j for proofs and a sampling of applications.) The basic idea 
behind the Lovasz local lemma is this: we have a finite family {Hi\ of "bad" events in a common 
probability space, and we are interested in the probability that none of these events occur, i.e., 
P(Plj-ffj). The "original" local lemma ^1] gives a sufficient condition for this probability to be 
strictly positive when most of the events Hi are independent, but with strong dependence allowed 
between some of the subsets of {H{\] for this reason it is formulated in terms of the dependency 
digraph ^3,^ for {Hi\. The version due to Erdos and Spencer [I5j (often referred to as "lopsided 
Lovasz local lemma") has the same content, but under the weaker condition that certain conditional 
probabilities involving the Hi and their complements are suitably bounded. More precisely: 

Theorem 3.1 (Erdos and Spencer |15p Let {Hi}'^^^ he a family of events in a common prob- 
ability space. Suppose that there exist a directed graph G = iV^E) with v{G) = n and real constants 
{'^iliLi, < rj < 1, such that, for any Y C V\N{i), 

F{H,\f]H])<r, n (1-r,). (3.3) 

Then 

n n 
1=1 i=l 

In other words, the event "none of the events Hi occur" holds with strictly positive probability. 

Consider now an acyclic digraph G = (y,E). Then, by defintion of e-admissibility, every 
probability measure € M.^ satisfies 

li[Ai\ {^A~)<e, vy C V\T\i). (3.5) 

We can rewrite ()3.5() in terms of the transitive closure graph G* as follows. Denote the out- 
neighborhood of a vertex i in G* by N*{i), and similarly for the closed out-neighborhood. Then 
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p.5|) becomes 

^{Ai\f]A^)< e, Vy C V*\N*{i). (3.6) 

Now let A be the maximum out-degree of G* , i.e., A := maxigy Then, provided that 

e < A^/(A + the events {Ai : i e V} wiU satisfy the condition (^3)) of Theorem lO with 

Tj = 1/(A + 1) for all i, for every /i E A4^. Using 1)3. 4() . we conclude that 

i^.(G) > (^1 - ' e<A^/(A + l)^+^ (3.7) 

Furthermore, F^i{G) > Fg{G) for e' < e because then we have A4f, C A4f. Thus, defining 
Fq := -Pa'^/(A+i)'^+i (G*), we can write 

Fe{G)>FG>^l-^^j , e<A^/(A + 1)^+1. (3.8) 
In fact. Lemma 13.41 in the next section can be used to obtain the exact expression 



i.e., Fg is equal precisely to the probability of 01"=? when the Ai are independent and f{Ai) = 
A^/(A + 1)^+1. 

An inspection of the form of (|3.9|) suggests the following strategy for exhibiting a phase tran- 
sition: We will consider a suitable parametrization Pc{n) of the (average) density^ of the random 
graph Gdin-jPcin)), withpc('^) — > as n — > oo, such that the graph is "sparse" for c < 1 and "dense" 
for c > 1. Furthermore, this change from "sparse" to "dense" will be accompanied by a phase tran- 
sition manifesting itself in the distinct large-n behavior of the size 7* of the largest RTC of a vertex 
in Gd{n,pc{n)) depending on whether c < 1, c = 1, or c > 1, respectively. (This phase transition 
was discovered and studied by Pittel and Tungol [HZj, and will play a key role in our proof.) Given 
a particular realization of G(i{n,Pcin)), 7* = A -|- 1. Defining random variables e„ := '&{'yn) 
have defined i}{x) := (x — l)^~^/x^ in order to avoid cluttered equations], we will end up with a 
sequence of e„-admissible PFM's, such that the asymptotic behavior of Fir^{Gd{n,pc{n)) will be 
different depending on whether c is above or below unity. 

Of course, the class of probability measures satisfying the condition (|3.5|) is much broader than 
Aif. In terms of Boolean networks, it describes the probabilistic failure model under which the 
conditional probability for a particular gate to fail, given that (any subset of) the gates preceding it 
have not failed, is at most e. (One can use the strategy of Lemma f2.1l to prove that the corresponding 
set of probability measures is compact.) In order to get a better grip on the e-admissible model, 
one has to make full use of its definition; this will be done in Section |1] using the machinery of 

^The density of a graph G is defined as d{G) := e{G)/v{G). Thus, we have for the expected density of the random 
graph G(n,p) 



The expected density of the random acyclic digraph Gd(n,p) is the same. 
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stochastic domination j261 1271 OH] . As far as the results in the next section are concerned, though, 
the condition l^'A.^ is all that is needed.^ Incidentally, it is possible to prove a more specialized 
version of the lopsided Lovasz local lemma which, among other things, gives a sufficient condition 
to have Eij > when the conditional probabilities 

F{Hi\ f~]Hjn (~] H]) (3.10) 

j&Y jeY' 

for all disjoint Y, Y' C V\N{i) are suitably bounded. Since this result is, strictly speaking, tangen- 
tial to the matter of this paper, we return to it in the Appendix. 



3.2 The main result 

Now we are in a position to state and prove the main result of this paper (the notation we use has 
been defined in the preceding section): 

Theorem 3.2 Consider all Boolean networks with n gates whose gate interconnection graphs are 
drawn from the random acyclic digraph Gdiri, cn~^ logn) . Let 7*(c) denote the size of the largest 
RTC of a vertex in Gd(n, cn""*^ logn). Define the sequence of random variables 

Fn{c) := F^(^*(c))(Gd(n,cn"Mogn)). (3.11) 



Then whp^ 

.exp^-^^) ifol 
The corresponding phase transition is illustrated in Fig. 



ifc<l 
F^{c) ^ F{c) ^ {_____ ^ ^ ^ (3.12) 



Remark 3.3 The proof of the theorem goes through [modulo obvious modifications involving the 
exponent in the c > 1 case of (|3.12() ] if, instead of ??(x), we use any nonnegative function f{x) that 
behaves like 1/x for large x. 

Proof. In order to carry on, we first need to gather some preliminary results. Once we have all 
the right pieces in place, the proof is actually surprisingly simple. 
The first result we need is the following exact formula for Fe(G): 

Lemma 3.4 For any acyclic digraph G = {V,E) with v{G) = n, F^{G) = (1 — e)". 

Proof. Let ii < i2 < ■ ■ ■ < in be an arrangement of the vertices of G according to some linear 
extension ^ of the partial order ^ defined in Section [2.11 Then, for each A; G [n], 

ij ^ik, l<j< k, (3.13) 

^Ironically enough, if at the very outset we were to use stochastic domination formalism to analyze /i( nigv'(G) ^0 ' 
fj. £ , we would not have been able to spot the role of the RTC in our development. 

^We follow standard practice of writing that a sequence of events E„ occurs whp (with high probability) if 
P{En) ^ 1 as n ^ oo. 



9 



1 c 

Figure 1: The phase transition of Theorem 13. 2t in the subcritical phase (0 < c < 1), the minimum 
probability of failure-free operation is zero; in the supercritical phase (c > 1), it is exp 
Note that F{c) approaches e~^^^ as c — > oo. 



so, by definition of e-admissibility, we have for any fi G M 

f 

Then 



G 



/^(^Wilfl =1^^.) 0<k<n-l. (3.14) 



m( n = (1 - ^^{A,,)) X n (i - I nil ^^.)) ^ (i - ^r- (3-15) 

The choice \x = ttY = Yiiev ^1 attains the bound in 1)3. 15(1 . and the lemma is proved. □ 

Next, we will need a result on the size of the largest reflexive, transitive closure of a vertex 
in the random acyclic digraph Gd{n,cn~^ logn). Pittel and Tungol j37j showed that the following 
phase transition takes place as c is varied: 

Lemma 3.5 For the random acyclic digraph Gd{n, cn~^ logn) one has the following: 
1- If c> 1, then there exists a positive constant A{c), such that 



lim I 

n— >oo 



1\ 2n log logn 



A(c)n', 

< = 1. 3.16 

log n 



c J c log n 
2. If c < 1, then for every x > 0, 

lim P{(1 - K)rf\ogn < 7* < n'^logn} = 1. (3.17) 

n^co 

The exact (and rather unwieldy) expressions of Lemma [3.5l are more than is needed for the purposes 
of the present proof. We will settle for a rather more prosaic asymptotic form that follows directly 
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from (|XT^ and (|TT7)) . Namely, whp 

7n(c) ~ < 



n'^ log n if < c < 1 
if c = 1 
ifol 



2?! log log n 
log n 



(3.18) 



n 



where, as usual, the notation a„ ~ 6„ means that a„ = 5„(l + o(l)). 

Next, we need asymptotics of the function i!){x) for large x. To that end, we write 



lim x-d^x) = lim ( 1 



x + l 



(3.19) 



In other words, ■d{x) ~ ^ as x — > oo. 

Finally, we will have use for the following two limits: 



Lemma 3.6 



lim I 1 

n— ►oo 

lim I 1 

n^oo 



1 



n'^ log n ^ 
log n 



0, 
0. 



< c < 1 



(3.20) 

11 , - (3.21) 

n log log n J 

Proof. We will prove (|3.2nj) : the same strategy will work for (|3.21|) as well. Let us assume that 
log denotes natural logarithms [otherwise we can multiply the second term in parentheses of 1)3. 2U|) 
by a suitable constant]. It suffices to show that 

1 



lim n log 1 — 

n^oo Y n'^ log n 

Using the inequality log x < x — 1 , we have for each n 



-oo. 



nlog I 1 



1 



n'^ log n 



< 



n 



l-c 



logn 



(3.22) 



(3.23) 



Given any K > 0, one can find large enough N such that the right-hand side of 1)3. 23() is less than 
—K for all n > N. Therefore the same holds for the left-hand side, and (|3.22l) is proved. □ 



The rest is fairly straightforward. Using Lemma 13.41 and (|3.19|) . we get 

l + o(l) 



F„(c) = (l-^?(7:(c)))" 



1 



e7n(c) 



(3.24) 



Now, using the asymptotics in 1)3. 18() . we obtain that whp 

r(i_i+£(ii)" ifc<i 

\ n'^ log n I 



1(1 



2n log log n / log n 

l+o(l) \n 
(l-c-i)ny' 



if c = 1 . 
if c> 1 



(3.25) 



Upon taking the limit as n — > oo of the expressions given in the right-hand side of 1)3. 25p and using 
Lemma 13.61 we obtain ()3.12l) . and the theorem is proved. □ 
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3.3 Discussion 



In retrospect, it is easy to see that Theorem 13 . 21 holds triviahy under the independent failure model. 
In order to appreciate nontrivial features that appear once we pass to Pippenger's model, we will 
invoke the game-theoretic interpretation given at the end of Section [2.21 

Consider a Programmer-Hacker game of the kind described in Section 12.21 Provided that the 
constant Eq picked by the Referee is sufficiently small, the Programmer has a good chance of 
winning if he sticks to the following strategy: Let no be the smallest integer and c the largest 
positive number, such that Eq < [e(l — l/c)no]~^ and cng^logno < 1. The Programmer generates 
a random acyclic digraph Grf(no, cHq ^ logno) and uses it to construct a Boolean network Nc(n) 
by adding variable inputs, assigning gate labels, and designating the output gate, possibly in a 
completely arbitrary fashion. He then hands this network to the Hacker. (Note that both the 
Programmer and the Hacker have all the information needed to determine which function / is 
computed by the network.) If c is large enough, then Theorem 13.21 thus guarantees the existence 
of some 6 < 1/2 such that, with probability 1 — o(l), the network generated by the Programmer 
will (eo, '^)-compute /, regardless of the Hacker's actions. More precisely, for any x, x' > 0, there 
exists large enough such that, for any c G (1, N/ log A^) and for all n > A^, 



Nc{n) ([e(l — l/c)n] (5)-computes / 



with 



(1 — (5) — exp 



> 1 - x'. (3.26) 



Provided that e ^/['^('^ — x > 1/2 and eo ^ [^(1 ~ l/c)?^o] ^ foi' some no > A^, the Programmer 
will win with probability at least 1 — x'. 



4 Pippenger's model and stochastic domination 

In Section [3.11 an argument based on the lopsided Lovasz local lemma (|15j. see also Theorem 13.11 
in this paper) allowed us to pinpoint the possibility for a phase transition in Pippenger's model on 
random graphs. In this section we show that the machinery of stochastic domination |26l 1271 128j 
enables us to carry out a more refined analysis of Pippenger's model. [We hasten to note that 
many of the issues which we will touch upon have, in fact, already been discussed by Pippenger in 
Ref. (34j, but without any systematic emphasis on stochastic domination.] 



4.1 Stochastic domination: the basics 

Once again, consider a finite acyclic digraph G = (y,E) along with the measurable space {^},J-), 
where Q. = and J- is the set of all subsets of Q,. Elements of Q are binary strings of length 
v{G); we will denote the ith component (bit) of a; G $7 by uj{i). The total ordering of B induces 
the following partial order of SI: 

UJI -< U2 < W2(i),Vi G V. (4.1) 

We say that a function / : $7 ^ M is increasing if ui ~< uj2 implies f{uji) < /(cj2)- An event 
H £ is called increasing if its indicator function, 1h, is increasing. Informally speaking, an event 
is increasing if its occurrence is unaffected by changing some bits from zero to one. Decreasing 
functions and events are defined in an obvious way. Given two probability measures //, on {Q,J^), 
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we will say that is stochastically dominated by i' (and write /x z^) if, for every increasing 
function /, < Ei/(/). As usual, the expectation E^(-) is defined by 

E^(/) := [ fdfi^Yl /(^)/^('^)- (4-2) 

Any probability measure on (fi,^) is equivalent to a family {Xj : i € y} of B- valued random 
variables via 

F{ /\{Xi = uj{i))) = H{uj), yujen (4.3) 

(also cf. Section |2.3() . We will say that the Xi have joint law fi if H4.3() holds. Then we have 
the following necessary and sufficient condition, due to Strassen j43j . for one measure to dominate 
another (this is, in fact, a type of result that is proved most naturally by means of the so-called 
coupling method; see the monograph by Lindvall for this as well as for many other useful 
applications of coupling). 

Lemma 4.1 Let /x, be probability measures on Then ^ ^ if <ind only if there exist 

families of random variables {Xi : i € V} and {Yi : i € V}, defined on a common probability space, 
with respective joint laws /x and v, such that Xi <Yi almost surely for each i . 

Next we need a sufficient condition for a given probability measure to dominate the Bernoulli 
product measure vr^. The following lemma is standard, and can be proved along the lines of Holley 
HSl and Preston pi: 

Lemma 4.2 Consider a family {Xi : i G V} of random variables with joint law /x. Suppose that 
there exists a total ordering <ofV such that, for any i G V and any two disjoint sets Y, Y' C 
with j < i for all j gY U Y' , we have 

¥^{Xi = l\ l\ {X, = 1) A /\ (X, = 0)) > r?, (4.4) 
whenever ¥^{/\-^y{^3 = 1) A /\j^Y'i^j = 0)) > 0- Then fJ. hs ■ 

Note that Lemma l4.'2l can also go in the other direction: that is, if instead we write the < sign in 
(|4.4|) . then we will have /x tt^- 

Before we go on, let us introduce one more definition. Given rj G (0, 1), let us define a class W,^ 
of probability measures on (17, J^) as follows. Let {Xi : i € V} be the family of B-valued random 
variables with joint law /x. Then /x G if, for any i G V and for any disjoint sets Y, Y' C V\N{i), 

^(X, = 1| /\ {X, = 1) A /\ {X, = 0)) > 7?, (4.5) 
whenever the event we condition on has positive probability. 
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4.2 Stochastic domination in Pippenger's model 

In this section we will use the machinery of stochastic domination to present a more delicate analysis 
of the e-admissible model of Pippenger. Let us fix (yet another!) acyclic digraph G = {V, E), which 
we will think of as a gate interconnection graph of a suitable class of Boolean networks equivalent 
modulo gate labels and the location of the output vertex (see Section l3.ll for details). Given a 
probability measure G '^f, let {Xi : i E V} be a family of IB-valued random variables with joint 
law fi. Let us define Xi := 1 — Xi for each i clearly, Xi is an indicator random variable for 

the event Ai = {gate i does not fail}. Let 77 := 1 — e. The joint law ji of {Xj} is such that, for any 
i GV and any two disjoint sets Y, Y' G y\r*(i), 

~^{Xi = 1\ f\ {Xj = 1) A /\ (X, = 0)) > r?, (4.6) 

jgy jeY' 

whenever the event we condition on has nonzero probability. Passing to the transitive closure graph 
G* = (y,E*), we can rewrite 1)4. 6() as follows: for any i gV and any disjoint Y,Y' C V\N*{i), 

~^{X, = 1\ /\ {X, = 1) A /\ {X, = 0)) > 7?. (4.7) 

jeY jev 

Comparing 1)4. 7(1 and 1)4. 5(1 . we see that fi e Mlf implies fl G . 

Let us show now that, for any /u G ■M'^, the corresponding /i satisfies /x ti"^- We can use the 
same strategy as in the proof of Lemma 13.41 Namely, if we rearrange the vertices of G according 
to some linear extension of the partial order ^, then fl is easily seen to satisfy the conditions of 
Lemma 14.21 and we obtain the claimed result. It follows directly from definitions that we have 
also fl :<s T^Y ■ We also point out that Strassen's theorem (Lemma 14.11 in this paper) can 
be used to give an amusing interpretation of Pippenger's model in terms of an intelligent agent 
("demon") who, when faced with a realization of {Xi} with joint law /i G Mf, can transform it 
into a realization of random variables i.i.d. according to vr^ by changing some bits from to 1, but 
none from 1 to 0. (The same observation has been first made by Pippenger j^, who substantiated 
it using a non-probabilistic result of Hwang [211 •) 

As part of our proof of Theorem 13. 2( we have obtained the exact formula i^e(G) = (1 — ey^^^ 
by arranging the vertices of G according to a linear extension of the partial order ^. The same 
conclusion can be easily reached once we have established that, for any G M^, we have /U 
irY [or, equivalently, that the corresponding fl G stochastically dominates vr^]; this extra 

information enables us to obtain many other useful estimates besides the one for Ff,[G). 

It is easy to see, for instance, that most of the "really interesting" events one would naturally 
associate with proper operation of the network (e.g., the event that the majority of the gates have 
not failed) are decreasing events. That is, if such an event occurs in a particular configuration 
u), then this event can be destroyed by introducing additional failed gates. The event that no 
gate fails is a particularly drastic example: it is destroyed if we change the status of even a single 
gate. Equivalently, we may pass to the corresponding probability measure fi. In that case, given 
a configuration lo G B^, the gate failures will correspond to zero bits of lo, with the nonzero 
bits indicating the gates that have not failed. Therefore we may consider increasing events if we 
agree to work with fl instead of fi. Using the stochastic domination properties of /i, we get that 
fl{H) > TT^ (H) for any increasing event H. 

As a simple example, consider the following set-up. Suppose we are given a network N whose 
output is the output of a gate that computes a Boolean function cp : —>■ M. Suppose that the 
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inputs to this gate come from the outputs of subnetworks Ni, . . . , (note that these subnetworks 
may, in general, share both gates and wires). Let M be the event that the majority of the gates 
in N have not failed, let Mj, 1 < i < d, be the event that the majority of the gates in Nj have not 
failed, and let L be the event that the output gate of N has not failed. Then L n HiLi implies 
M, so that 

P(M) > P(L n fl^^^ Mi) = P(L| fl^^^ Mi)P(f|^^^ Mi). (4.8) 
Suppose that the underlying probability measure ^ is e-admissible. Then 

P(L|f|^^^M,) >r?. (4.9) 

Likewise by e-admissibility, fi tt^ (see Section [2.21 for this notation). Therefore, since an inter- 
section of increasing events is increasing, we have 

The right-hand side of l|4.10j) can be bounded from below by means of the FKG inequality |18j 
(proved earlier by Harris ^20^ in the context of percolation) to give 

d 

^A(^^^,M^)>\{<'m■ (4-11) 

Let a be the number of gates in the smallest of the subnetworks Ni, . . . , N^. Then, assuming that 
e < 1/2, Azuma-Hoeffding inequality |3I22] gives 

7r|)'"(Mi) > l-e-"(^''-2)'. (4.12) 

Putting everything together, we get 

P(M) >r/(l-e-''(^''-2)')^ (4.13) 

This use of the FKG inequality is similar to that in Feder , whose work was concerned with the 
depth and reliability bounds for reliable computation of Boolean functions under the independent 
failure model. 

At this point we also mention another PFM discussed by Pippenger in Ref. — namely, the 
so-called e-majorized model. Under this model, a network N is mapped to the set of all probability 
measures /j on B^'^ that are stochastically dominated by the Bernoulli product measure vr^. It 
follows easily from the discussion above that the e-majorized model is more stringent than the e- 
admissible model. As an example of its use, we can mention the work of Dobrushin and Ortyukov 
jl2j . where a result proven for the e-majorized model automatically carries over to the e-independent 
one. 

5 Closing remarks and future directions 

In this paper we have showed that a phase transition is possible in the e-admissible PFM of Pip- 
penger as soon as random graphs show up in the picture. In doing so, we have barely scratched 
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the surface of a wonderfully rich subject — namely, the statistical mechanics of multicomponent 
random systems on directed graphs. Most of the work connected to phase transitions in large 
combinatorial structures has been done in the context of undirected graphs, since the methods 
of statistical mechanics applicable to the study of combinatorial structures have been originally 
developed in that context as well. 

For instance, the independent-set polynomial of a simple undirected graph \17\ \21\ I41j can also 
be viewed as a partition function of a repulsive lattice gas |42j, and the powerful machinery of 
cluster expansions ^0] developed in the latter setting can also be applied quite successfully to 
the former; the reader is encouraged to consult a recent paper by Scott and Sokal PH] for an 
exposition of these matters from the viewpoint of both statistical mechanics and graph theory. In 
the future we would like to study applications of statistical mechanics to combinatorial structures 
with directionality. So far, very few results along these lines have appeared; the papers of Whittle 
j45| I4()j are among the few examples known to the present author where a partition function is 
constructed for a class of statistical-mechanical models on directed graphs. The relative dearth of 
applications of statistical mechanics to structures with directionality is due to the fact that, once 
directionality is introduced, the symmetry needed for the lattice-gas formalism is destroyed, and 
it is not immediately evident how one could relate combinatorial properties of directed graphs to 
mathematical objects of statistical mechanics. (Incidentally, this very point has also been brought 
up by Scott and Sokal [39].) 



Appendix 

Our goal in this appendix is to prove a theorem that can be thought of as a specialization of the 
lopsided Lovasz local lemma of Erdos and Spencer (see also Theorem 13. 1|) to families of random 
variables whose joint laws are elements of W^. Both the theorem and its proof go very much along 
the lines a similar result of Liggett, Schonmann, and Stacey |27j . except that theirs was formulated 
for undirected graphs. 

Theorem A.l Let G = (y,E) be a directed acyclic graph, in which every vertex has out-degree at 
most A > 1. Let fie with e := I - r] < A^/(A + l)(^+i). Then ^ vr^, where 

'>=f'-|^)(l-(^^)*""*"). (A.1) 



Proof. First we need a lemma. 



Lemma A. 2 Let G = {y,E) satisfy the conditions of Theorem \A . 1\ Given r] S (0,1), consider 
fj- G VV^- Suppose that there exist constants a, A € (0, 1), such that 

e< (l-a)(l-A)^, (A.2) 



e<(l-a)a^. (A.3) 

Consider a family {Xi : i G V} of random variables with joint law fi, and let {Yi : i G V} be a 
family of random variables, independent of {Xi} and with joint law tt]^. Let Zi := XiYi for each 
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i €zV. Then, for each i eV, each Y C and each z G B'^I, we have 

\y\ 



,)) > «A, 



(A.4) 



where the ij are elements ofY. 

Remark A. 3 The corresponding theorem of Liggett, Schonmann, and Stacey is formulated 
in terms of an undirected graph G, with A being the maximum degree of a vertex. Therefore they 
need to impose an additional condition, namely that i is adjacent to at most A — 1 vertices in Y . 
As a consequence, one has to make the replacement A A — 1, e.g., in ()A.1|) . (|A.2|) and ()A.3|) . 
However, because here we deal with directed graphs and A is the maximum out-degree of a vertex, 
there are automatically no more than A vertices j in G with {i,j) € E. 

Proof. Note that ()A.4|) is equivalent to 

due to independence of {Xi} and {Yi\ and to the fact that A > 0. We will proceed by proving 
HA.5|) by induction on \Y\. 

Suppose first that Y = 0. Then (|A.5|) is simply the statement that P(Xj = 1) > a. Now, 
¥{Xi = 1) > 77 because ^ € VV^, and r/ > a by ()A.3|) . Thus suppose that ()A.5|) holds for all 
Y C with \Y\ < J, where J > 1. We will prove that it also holds for \Y\ = J. 



P(X. = l|/\ 



(Zi- 



'■3)) > « 



(A.5) 



Fix Y = {ii, . . . ,ij} and z € 

Mo 
Ml 
M 

Let us also define the events 



. We write 1" as a disjoint union Mq U Mi U M, where 



{ijA <j<J- {i,ij] 
<j<J-- {i,ij] 
Y\{MoUMi). 



E E and Zj 
E E and Zj 



0} 
1} 



^0 




= : 


ij G Mo} 


Bo 


■■= {Yi^ 


= : 


ij G Mo} 


Ai 


■■= {x,^ 


= 1 


ij G Ml} 


A 




= ^3 


: ij G N} 



Now, for any j G V, Zj = 1 is by definition equivalent to both Xj 
{Xj} and {Yj} are independent. Therefore we can write 

J 



1 and Yo- 



Xi 



Zi 



;)) 



Now 



'{Xi = i\AonAinA) = 1 



> 1 



Xi = l\Ao nAiHA) 
•{Xi = 0\Ao nAiHA) 

'{Xi = o,AonAinA) 
p(^o n Ai n^) 

F{Xi = 0,A) 



(A.6) 
(A.7) 
(A.8) 

(A.9) 
(A.IO) 
(A.ll) 
(A.12) 

1. Furthermore, 
(A.13) 



F{Bo nAiHA) 
P{Xi = 0\A) 
F{BonAi\A)' 



(A.14) 
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Since ^ E for all ij G M, the numerator is at most e. The denominator is equal to (1 — 

A)l^olp(Aij^). Assume that Ml = {A;i, ... s = |Mi I . Then, 

F{AM) = n/\ljXk, = l)\A) 

'-^ i 
= P(X,, = l\A)l[F{X,,^^ = 1\A, f\^^^{X,^ = 1)) 

£=1 

> al*^^l, (A.15) 

where in the last step we have applied the inductive hypothesis to each of the terms in the product. 
Therefore 

P(x, = i|ylo n ^1 n ^) > 1 - ^^_^^|',^|^|,,^| - (A.16) 

Since |Mo| + |Afi| < A by hypothesis, and e/(l — a) < < 1 by HA.3|) . we have 

/ ^ N (|A/o| + |Afi|)/A 
(1 _ X)\Mo\a\M,\ > ( _J_ \ > (A.17) 

\1 — a J 1 — a 

Therefore the right-hand side of ()A.16|) is at least 1 — e/[e/(l — a)] = a, and the lemma is proved. □ 

Now let {Xi}^ {^}) and {Zi\ be as in Lemma IA.2I Let v be the joint law of {Zi}. By 
construction, Zi < XiYi for each i € so /i by Lemma l4.ll We now show that u tt^^. 

Let < be an arbitrary total ordering of V. Then Lemma lA. 21 and Lemma 14.21 implv that v vr^^. 
Thus fi T^aX- 

To conclude the proof, suppose that 

<rl/(A + l) 

« = 1-Ta77aTTT and A = 1 - (eA)i/{A+i) . (a.19) 



Let 



A^/(^+i) 

Then (I - a)(l - A)^ = e, which yields (|X2)) . Condition (|A.18|) is equivalent to 

A A/{A+1) 
^1/(A+1) < AA > 

~ A+ 1 ^ ^ 

which, when substitued into ()A.19|) . yields 

1 1 

a > 1 - and A > . (A.21) 

A + 1 ~ A + 1 ^ ^ 

This shows that the choice we have made in (|A.19|) leads to a, A € [0, 1], and that 1 — A > a. The 
latter inequality implies (|A.3|) . Therefore, by Lemma lA. 21 fi tt^A' theorem is proved. □ 

It is important to mention that Theorem lA.ll is useful only when G is not transitively closed, 
i.e., when G E and (j, k) ^ E does not imply (i, k) G E. Otherwise one can partially order the 
vertices of G by defining i ^ j \i G E for distinct i,j G V, and i ^ i for each i £ V. As usual, 
let ii < . . . < z^(c') be a total order of V according to some linear extension of ^. Thus, for any ij, 
all the ifc with < ij and distinct from ij are not in N{ij). Therefore we can apply Lemma 14.21 
directly to any fx G to conclude that /U ^r^- This is, in fact, precisely the case we have dealt 
with in this paper — namely, when G is a transitive closure of some other acyclic digraph Gq. 
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